============================= test session starts ============================== platform linux -- Python 3.7.5, pytest-5.4.3, py-1.11.0, pluggy-0.13.1 rootdir: /home/jenkins/mindspore/testcases/testcases/tests/st/heterogeneous, inifile: /home/jenkins/sault/virtual_test/virtualenv_003/sault/config/pytest.ini plugins: xdist-1.32.0, forked-1.4.0 [INFO] ATRACE(61248,python3.7):2024-01-10-09:35:21.062.178 [trace_attr.c:105](tid:61248) platform is 1. [INFO] ATRACE(61248,python3.7):2024-01-10-09:35:21.062.337 [trace_recorder.c:114](tid:61248) use root path: /home/jenkins/ascend/atrace [INFO] ATRACE(61248,python3.7):2024-01-10-09:35:21.062.362 [trace_signal.c:133](tid:61248) register signal handler for signo 2 succeed. [INFO] ATRACE(61248,python3.7):2024-01-10-09:35:21.062.373 [trace_signal.c:133](tid:61248) register signal handler for signo 15 succeed. [INFO] RUNTIME(61248,python3.7):2024-01-10-09:35:21.412.360 [runtime.cc:1159] 61248 GetAicoreNumByLevel: workingDev_=0 [INFO] RUNTIME(61248,python3.7):2024-01-10-09:35:21.412.429 [runtime.cc:4719] 61248 GetVisibleDevices: ASCEND_RT_VISIBLE_DEVICES param was not set collected 2 items test_heter_control.py [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.866.564 [process_mode_manager.cpp:109][OpenProcess][tid:61248] [ProcessModeManager] enter into open process deviceId[2] rankSize[0] [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.867.346 [process_mode_manager.cpp:379][InitTsdClient][tid:61248] [TsdClient] deviceId[2] begin to init hdc client [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.867.474 [version_verify.cpp:34][SetVersionInfo][tid:61248] VersionVerify: send client version to server [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.867.538 [version_verify.cpp:50][SetVersionInfo][tid:61248] send feature_info:{msg_type:35, features:{check before send aicpu package,}} [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.867.553 [version_verify.cpp:50][SetVersionInfo][tid:61248] send feature_info:{msg_type:37, features:{check before send open qs message,}} [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.867.830 [version_verify.cpp:66][PeerVersionCheck][tid:61248] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.867.845 [version_verify.cpp:87][ParseVersionInfo][tid:61248] VersionVerify: pass client version info success [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.867.854 [hdc_client.cpp:276][CheckHdcConnection][tid:61248] Service[2] create hdc success [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.867.866 [version_verify.cpp:120][SpecialFeatureCheck][tid:61248] VersionVerify: new type[35], supported [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.867.909 [process_mode_manager.cpp:748][GetDeviceCheckCode][tid:61248] [TsdClient][deviceId=2] [sessionId=1] wait package info respond [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.868.016 [process_mode_manager.cpp:379][InitTsdClient][tid:61248] [TsdClient] deviceId[2] begin to init hdc client [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.868.118 [version_verify.cpp:34][SetVersionInfo][tid:61248] VersionVerify: send client version to server [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.868.131 [version_verify.cpp:50][SetVersionInfo][tid:61248] send feature_info:{msg_type:35, features:{check before send aicpu package,}} [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.868.141 [version_verify.cpp:50][SetVersionInfo][tid:61248] send feature_info:{msg_type:37, features:{check before send open qs message,}} [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.868.338 [version_verify.cpp:66][PeerVersionCheck][tid:61248] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.868.350 [version_verify.cpp:87][ParseVersionInfo][tid:61248] VersionVerify: pass client version info success [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.868.358 [hdc_client.cpp:276][CheckHdcConnection][tid:61248] Service[2] create hdc success [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.868.369 [process_mode_manager.cpp:426][ConstructOpenMsg][tid:61248] [TsdClient] tsd get process sign successfully, procpid[61248] signSize[48] [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.868.401 [version_verify.cpp:112][SpecialFeatureCheck][tid:61248] VersionVerify: previous type[6], supported [INFO] TDT(61248,python3.7):2024-01-10-09:35:26.868.422 [process_mode_manager.cpp:126][OpenProcess][tid:61248] [ProcessModeManager] deviceId[2] sessionId[1] rankSize[0], wait sub process start respond [INFO] TDT(61248,python3.7):2024-01-10-09:35:27.769.072 [stub_process_mode_nowin.cpp:63][ProcessQueueForMdc][tid:61248] [TsdClient] it is unnecessary of current mode[0] chiptype[1] to grant queue auth to aicpusd [INFO] TDT(61248,python3.7):2024-01-10-09:35:27.769.123 [stub_process_mode_nowin.cpp:101][OpenInHost][tid:61248] enter into OpenInHost deviceid[2] [INFO] TDT(61248,python3.7):2024-01-10-09:35:27.769.135 [stub_process_mode_nowin.cpp:105][OpenInHost][tid:61248] host cpu not support [INFO] TDT(61248,python3.7):2024-01-10-09:35:27.769.143 [process_mode_manager.cpp:156][OpenProcess][tid:61248] [TsdClient][deviceId=2] [sessionId=1] start hccp and computer process success [INFO] RUNTIME(61248,python3.7):2024-01-10-09:35:27.771.527 [device.cc:340] 61248 Init: isDoubledie:0, topologytype:0 [INFO] RUNTIME(61248,python3.7):2024-01-10-09:35:27.784.262 [npu_driver.cc:5428] 63238 GetDeviceStatus: GetDeviceStatus status=1. [INFO] ATRACE(61248,python3.7):2024-01-10-09:35:27.784.310 [atrace_api.c:28](tid:61248) AtraceCreate start [INFO] ATRACE(61248,python3.7):2024-01-10-09:35:27.784.423 [trace_rb_log.c:84](tid:61248) [RUNTIME_ATRACE_DEV2_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(61248,python3.7):2024-01-10-09:35:27.784.440 [atrace_api.c:32](tid:61248) AtraceCreate end [INFO] TDT(61248,python3.7):2024-01-10-09:35:27.784.465 [client_manager.cpp:157][SetProfilingCallback][tid:61248] [TsdClient] set profiling callback success [TRACE] GE(61248,python3.7):2024-01-10-09:35:27.918.824 [status:INIT] [ge_api.cc:144]61248 GEInitializeImpl:GEInitialize start [INFO] PROFILING(61248,python3.7):2024-01-10-09:35:28.146.144 [msprofiler_impl.cpp:156] >>> (tid:61248) ProfNotifySetDevice called, is open: 1, devId: 2 [INFO] PROFILING(61248,python3.7):2024-01-10-09:35:28.146.301 [platform.cpp:38] >>> (tid:61248) Profiling platform version: 1.0. [INFO] PROFILING(61248,python3.7):2024-01-10-09:35:28.146.319 [ai_drv_dev_api.cpp:384] >>> (tid:61248) Succeeded to DrvGetApiVersion version: 0x72313 [TRACE] GE(61248,python3.7):2024-01-10-09:35:28.196.600 [status:RUNNING] [ge_api.cc:211]61248 GEInitializeImpl:Initializing environment [INFO] GE(61248,python3.7):2024-01-10-09:35:28.196.686 [gelib.cc:98][EVENT]61248 Initialize:[GEPERFTRACE] GE Init Start [INFO] GE(61248,python3.7):2024-01-10-09:35:28.196.995 [gelib.cc:307][EVENT]61248 SystemInitialize:Online infer init GELib success, device id :2 [INFO] DVPP(61248,python3.7):2024-01-10-09:35:28.554.617 [dvpp_engine.cc:41][ENGINE][Initialize:41][tid:61248]dvpp engine do not support [INFO] TUNE(61248,python3.7):2024-01-10-09:35:28.559.803 [cann_kb_pyfunc_mgr.cpp:72][CANNKB][Tid:61248]"CannKbPyfuncMgr: Enter PyObjectInit, reference_ is 0!" [INFO] TUNE(61248,python3.7):2024-01-10-09:35:28.559.852 [handle_manager.cpp:115][CANNKB][Tid:61248]"Start to run init functions to load dynamic python lib!" [INFO] TUNE(61248,python3.7):2024-01-10-09:35:28.559.906 [handle_manager.cpp:407][CANNKB][Tid:61248]"Init functions of loading dynamic python lib end!" [INFO] TUNE(61248,python3.7):2024-01-10-09:35:28.559.917 [cann_kb_pyfunc_mgr.cpp:24][CANNKB][Tid:61248]"CANN_KB_Py has already been initialized." [INFO] TUNE(61248,python3.7):2024-01-10-09:35:28.560.235 [cann_kb_pyfunc_mgr.cpp:117][CANNKB][Tid:61248]"CannKbPyfuncMgr: Run PyObjectInit successfully!" [INFO] HCCL(61248,python3.7):2024-01-10-09:35:41.358.949 [plugin_manager.cc:42][61248]hcom running normal mode. [INFO] DVPP(61248,python3.7):2024-01-10-09:35:41.359.691 [dvpp_engine.cc:92][ENGINE][GetOpsKernelInfoStores:92][tid:61248]dvpp ops kernel info store do not support [INFO] DVPP(61248,python3.7):2024-01-10-09:35:41.359.871 [dvpp_engine.cc:69][ENGINE][GetGraphOptimizerObjs:69][tid:61248]dvpp graph optimizer do not support [INFO] DVPP(61248,python3.7):2024-01-10-09:35:41.882.330 [dvpp_ops_kernel_builder.cc:48][ENGINE][Initialize:48][tid:61248]dvpp ops kernel builder do not support [INFO] GE(61248,python3.7):2024-01-10-09:35:41.891.449 [gelib.cc:169][EVENT]61248 Initialize:[GEPERFTRACE] The time cost of GELib::Initialize is [13694660] micro second. [TRACE] GE(61248,python3.7):2024-01-10-09:35:41.984.167 [status:STOP] [ge_api.cc:255]61248 GEInitializeImpl:GEInitialize finished [TRACE] GE(61248,python3.7):2024-01-10-09:35:41.984.343 [status:INIT] [ge_api.cc:398]61248 Session:Start to construct session. [TRACE] GE(61248,python3.7):2024-01-10-09:35:41.984.362 [status:RUNNING] [ge_api.cc:408]61248 Session:Creating session [INFO] GE(61248,python3.7):2024-01-10-09:35:41.984.893 [graph_var_manager.cc:1445][EVENT]61248 SetMemoryMallocSize:Total memory size is 34359738368 [INFO] GE(61248,python3.7):2024-01-10-09:35:41.984.911 [graph_var_manager.cc:1424][EVENT]61248 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] PROFILING(61248,python3.7):2024-01-10-09:35:41.985.339 [msprofiler_impl.cpp:156] >>> (tid:61248) ProfNotifySetDevice called, is open: 1, devId: 2 [TRACE] GE(61248,python3.7):2024-01-10-09:35:41.986.217 [status:RUNNING] [ge_api.cc:411]61248 Session:Session id is 0 [TRACE] GE(61248,python3.7):2024-01-10-09:35:41.986.238 [status:STOP] [ge_api.cc:420]61248 Session:Session Constructor finished TotalTime = 15.2202, [20] [parse]: 0.0133049 [symbol_resolve]: 0.00127211, [1] [Cycle 1]: 0.0012024, [1] [resolve]: 0.00117763 [combine_like_graphs]: 1.46002e-06 [graph_reusing]: 4.05003e-06 [meta_unpack_prepare]: 9.312e-05 [pre_cconv]: 1.92701e-05 [abstract_specialize]: 0.0373364 [pack_expand]: 2.175e-05 [auto_monad]: 0.00017609 [inline]: 2.51998e-06 [pre_auto_parallel]: 1.848e-05 [pipeline_split]: 3.20002e-06 [optimize]: 0.0143536, [35] [py_interpret_to_execute]: 3.981e-05 [rewriter_before_opt_a]: 0.00011955 [opt_a]: 0.0123769, [2] [Cycle 1]: 0.00389637, [30] [expand_dump_flag]: 7.07997e-06 [switch_simplify]: 0.00011591 [a_1]: 0.00083728 [recompute_prepare]: 1.545e-05 [updatestate_depend_eliminate]: 2.089e-05 [updatestate_assign_eliminate]: 1.154e-05 [updatestate_loads_eliminate]: 9.92999e-06 [parameter_eliminate]: 8.47003e-06 [a_2]: 0.00018198 [accelerated_algorithm]: 1.349e-05 [pynative_shard]: 2.61003e-06 [auto_parallel]: 8.22998e-06 [parallel]: 2.25e-05 [merge_comm]: 1.223e-05 [allreduce_fusion]: 2.39996e-06 [virtual_dataset]: 8.28999e-06 [get_grad_eliminate_]: 6.43998e-06 [virtual_output]: 6.38998e-06 [merge_forward]: 1.39e-05 [cell_reuse_recompute_pass]: 7.89994e-07 [cell_reuse_handle_not_recompute_node_pass]: 1.943e-05 [meta_fg_expand]: 1.036e-05 [after_resolve]: 9.29e-06 [a_after_grad]: 8.48999e-06 [renormalize]: 0.00217851 [real_op_eliminate]: 1.365e-05 [auto_monad_grad]: 9.59e-06 [auto_monad_eliminator]: 2.955e-05 [cse]: 6.599e-05 [a_3]: 6.442e-05 [Cycle 2]: 0.00065021, [30] [expand_dump_flag]: 1.70997e-06 [switch_simplify]: 1.191e-05 [a_1]: 0.00011098 [recompute_prepare]: 1.07e-05 [updatestate_depend_eliminate]: 1.262e-05 [updatestate_assign_eliminate]: 9.69996e-06 [updatestate_loads_eliminate]: 9.25997e-06 [parameter_eliminate]: 3.10998e-06 [a_2]: 0.00013055 [accelerated_algorithm]: 1.079e-05 [pynative_shard]: 1.22999e-06 [auto_parallel]: 4.83997e-06 [parallel]: 5.69003e-06 [merge_comm]: 3.28e-06 [allreduce_fusion]: 1.75001e-06 [virtual_dataset]: 8.57003e-06 [get_grad_eliminate_]: 6.07001e-06 [virtual_output]: 5.78997e-06 [merge_forward]: 1.118e-05 [cell_reuse_recompute_pass]: 4.10015e-07 [cell_reuse_handle_not_recompute_node_pass]: 1.656e-05 [meta_fg_expand]: 7.57998e-06 [after_resolve]: 7.12002e-06 [a_after_grad]: 8.35001e-06 [renormalize]: 7.00238e-08 [real_op_eliminate]: 5.89003e-06 [auto_monad_grad]: 2.31003e-06 [auto_monad_eliminator]: 2.065e-05 [cse]: 4.486e-05 [a_3]: 5.814e-05 [py_interpret_to_execute_after_opt_a]: 2.858e-05 [slice_cell_reuse_recomputed_activation]: 2.93e-06 [rewriter_after_opt_a]: 0.0004197 [convert_after_rewriter]: 2.177e-05 [order_py_execute_after_rewriter]: 1.267e-05 [opt_b]: 0.00028974, [1] [Cycle 1]: 0.00028348, [7] [b_1]: 0.00016132 [b_2]: 8.78004e-06 [updatestate_depend_eliminate]: 1.207e-05 [updatestate_assign_eliminate]: 9.50001e-06 [updatestate_loads_eliminate]: 8.96002e-06 [renormalize]: 5.60016e-07 [cse]: 4.937e-05 [cconv]: 0.00022347 [opt_after_cconv]: 0.00022106, [1] [Cycle 1]: 0.0002154, [7] [c_1]: 3.623e-05 [parameter_eliminate]: 7.84005e-06 [updatestate_depend_eliminate]: 1.629e-05 [updatestate_assign_eliminate]: 1.169e-05 [updatestate_loads_eliminate]: 1.204e-05 [cse]: 9.68101e-05 [renormalize]: 4.4005e-07 [remove_dup_value]: 6.565e-05 [tuple_transform]: 0.00011635, [1] [Cycle 1]: 0.00011135, [3] [d_1]: 6.774e-05 [d_2]: 3.006e-05 [renormalize]: 2.3999e-07 [add_cache_embedding]: 2.914e-05 [add_recomputation]: 8.804e-05 [cse_after_recomputation]: 6.173e-05, [1] [Cycle 1]: 5.675e-05, [1] [cse]: 5.08e-05 [environ_conv]: 1.866e-05 [label_micro_interleaved_index]: 2.49001e-06 [label_fine_grained_interleaved_index]: 2.54001e-06 [assign_add_opt]: 1.09e-06 [slice_recompute_activation]: 2.85998e-06 [micro_interleaved_order_control]: 2.53e-06 [full_micro_interleaved_order_control]: 2.81999e-06 [comp_comm_scheduling]: 2.31003e-06 [reorder_send_recv_between_fp_bp]: 1.92999e-06 [comm_op_add_attrs]: 1.10996e-06 [add_comm_op_reuse_tag]: 9.39996e-07 [overlap_opt_shard_in_pipeline]: 1.27999e-06 [grouped_pairwise_exchange_alltoall]: 1.31998e-06 [overlap_recompute_and_grad_model_parallel]: 2.06003e-06 [overlap_grad_matmul_and_grad_allreduce]: 7.40052e-07 [split_matmul_comm_elemetwise]: 2.55002e-06 [split_layernorm_comm]: 2.16998e-06 [process_send_recv_for_ge]: 3.93996e-06 [handle_group_info]: 1.15001e-06 [auto_monad_reorder]: 5.91e-05 [get_jit_bprop_graph]: 4.70027e-07 [eliminate_special_op_node]: 0.00062405 [validate]: 6.161e-05 [distribtued_split]: 1.19e-06 [task_emit]: 15.1526 [execute]: 7.5e-06 Sums parse : 0.013305s : 0.09% symbol_resolve.resolve : 0.001178s : 0.01% combine_like_graphs : 0.000001s : 0.00% graph_reusing : 0.000004s : 0.00% meta_unpack_prepare : 0.000093s : 0.00% pre_cconv : 0.000019s : 0.00% abstract_specialize : 0.037336s : 0.25% pack_expand : 0.000022s : 0.00% auto_monad : 0.000176s : 0.00% inline : 0.000003s : 0.00% pre_auto_parallel : 0.000018s : 0.00% pipeline_split : 0.000003s : 0.00% optimize.py_interpret_to_execute : 0.000040s : 0.00% optimize.rewriter_before_opt_a : 0.000120s : 0.00% optimize.opt_a.expand_dump_flag : 0.000009s : 0.00% optimize.opt_a.switch_simplify : 0.000128s : 0.00% optimize.opt_a.a_1 : 0.000948s : 0.01% optimize.opt_a.recompute_prepare : 0.000026s : 0.00% optimize.opt_a.updatestate_depend_eliminate : 0.000034s : 0.00% optimize.opt_a.updatestate_assign_eliminate : 0.000021s : 0.00% optimize.opt_a.updatestate_loads_eliminate : 0.000019s : 0.00% optimize.opt_a.parameter_eliminate : 0.000012s : 0.00% optimize.opt_a.a_2 : 0.000313s : 0.00% optimize.opt_a.accelerated_algorithm : 0.000024s : 0.00% optimize.opt_a.pynative_shard : 0.000004s : 0.00% optimize.opt_a.auto_parallel : 0.000013s : 0.00% optimize.opt_a.parallel : 0.000028s : 0.00% optimize.opt_a.merge_comm : 0.000016s : 0.00% optimize.opt_a.allreduce_fusion : 0.000004s : 0.00% optimize.opt_a.virtual_dataset : 0.000017s : 0.00% optimize.opt_a.get_grad_eliminate_ : 0.000013s : 0.00% optimize.opt_a.virtual_output : 0.000012s : 0.00% optimize.opt_a.merge_forward : 0.000025s : 0.00% optimize.opt_a.cell_reuse_recompute_pass : 0.000001s : 0.00% optimize.opt_a.cell_reuse_handle_not_recompute_node_pass : 0.000036s : 0.00% optimize.opt_a.meta_fg_expand : 0.000018s : 0.00% optimize.opt_a.after_resolve : 0.000016s : 0.00% optimize.opt_a.a_after_grad : 0.000017s : 0.00% optimize.opt_a.renormalize : 0.002179s : 0.01% optimize.opt_a.real_op_eliminate : 0.000020s : 0.00% optimize.opt_a.auto_monad_grad : 0.000012s : 0.00% optimize.opt_a.auto_monad_eliminator : 0.000050s : 0.00% optimize.opt_a.cse : 0.000111s : 0.00% optimize.opt_a.a_3 : 0.000123s : 0.00% optimize.py_interpret_to_execute_after_opt_a : 0.000029s : 0.00% optimize.slice_cell_reuse_recomputed_activation : 0.000003s : 0.00% optimize.rewriter_after_opt_a : 0.000420s : 0.00% optimize.convert_after_rewriter : 0.000022s : 0.00% optimize.order_py_execute_after_rewriter : 0.000013s : 0.00% optimize.opt_b.b_1 : 0.000161s : 0.00% optimize.opt_b.b_2 : 0.000009s : 0.00% optimize.opt_b.updatestate_depend_eliminate : 0.000012s : 0.00% optimize.opt_b.updatestate_assign_eliminate : 0.000010s : 0.00% optimize.opt_b.updatestate_loads_eliminate : 0.000009s : 0.00% optimize.opt_b.renormalize : 0.000001s : 0.00% optimize.opt_b.cse : 0.000049s : 0.00% optimize.cconv : 0.000223s : 0.00% optimize.opt_after_cconv.c_1 : 0.000036s : 0.00% optimize.opt_after_cconv.parameter_eliminate : 0.000008s : 0.00% optimize.opt_after_cconv.updatestate_depend_eliminate : 0.000016s : 0.00% optimize.opt_after_cconv.updatestate_assign_eliminate : 0.000012s : 0.00% optimize.opt_after_cconv.updatestate_loads_eliminate : 0.000012s : 0.00% optimize.opt_after_cconv.cse : 0.000097s : 0.00% optimize.opt_after_cconv.renormalize : 0.000000s : 0.00% optimize.remove_dup_value : 0.000066s : 0.00% optimize.tuple_transform.d_1 : 0.000068s : 0.00% optimize.tuple_transform.d_2 : 0.000030s : 0.00% optimize.tuple_transform.renormalize : 0.000000s : 0.00% optimize.add_cache_embedding : 0.000029s : 0.00% optimize.add_recomputation : 0.000088s : 0.00% optimize.cse_after_recomputation.cse : 0.000051s : 0.00% optimize.environ_conv : 0.000019s : 0.00% optimize.label_micro_interleaved_index : 0.000002s : 0.00% optimize.label_fine_grained_interleaved_index : 0.000003s : 0.00% optimize.assign_add_opt : 0.000001s : 0.00% optimize.slice_recompute_activation : 0.000003s : 0.00% optimize.micro_interleaved_order_control : 0.000003s : 0.00% optimize.full_micro_interleaved_order_control : 0.000003s : 0.00% optimize.comp_comm_scheduling : 0.000002s : 0.00% optimize.reorder_send_recv_between_fp_bp : 0.000002s : 0.00% optimize.comm_op_add_attrs : 0.000001s : 0.00% optimize.add_comm_op_reuse_tag : 0.000001s : 0.00% optimize.overlap_opt_shard_in_pipeline : 0.000001s : 0.00% optimize.grouped_pairwise_exchange_alltoall : 0.000001s : 0.00% optimize.overlap_recompute_and_grad_model_parallel : 0.000002s : 0.00% optimize.overlap_grad_matmul_and_grad_allreduce : 0.000001s : 0.00% optimize.split_matmul_comm_elemetwise : 0.000003s : 0.00% optimize.split_layernorm_comm : 0.000002s : 0.00% optimize.process_send_recv_for_ge : 0.000004s : 0.00% optimize.handle_group_info : 0.000001s : 0.00% auto_monad_reorder : 0.000059s : 0.00% get_jit_bprop_graph : 0.000000s : 0.00% eliminate_special_op_node : 0.000624s : 0.00% validate : 0.000062s : 0.00% distribtued_split : 0.000001s : 0.00% task_emit : 15.152568s : 99.61% execute : 0.000007s : 0.00% Time group info: ------[substitution.] 0.001622 245 3.85% : 0.000062s : 12: substitution.arithmetic_simplify 1.04% : 0.000017s : 2: substitution.cast_eliminate 2.77% : 0.000045s : 4: substitution.compare_switch_simplify 1.83% : 0.000030s : 15: substitution.exchange_switch_depend_value 0.16% : 0.000003s : 4: substitution.float_depend_g_call 58.48% : 0.000949s : 6: substitution.getattr_setattr_resolve 1.58% : 0.000026s : 27: substitution.graph_param_transform 0.32% : 0.000005s : 4: substitution.incorporate_call 0.14% : 0.000002s : 4: substitution.incorporate_call_switch 22.14% : 0.000359s : 13: substitution.inline 0.28% : 0.000005s : 2: substitution.less_batch_normalization 2.59% : 0.000042s : 20: substitution.meta_unpack_prepare 0.28% : 0.000005s : 6: substitution.partial_eliminate 0.38% : 0.000006s : 27: substitution.partial_unused_args_eliminate 0.23% : 0.000004s : 28: substitution.remove_not_recompute_node 0.78% : 0.000013s : 16: substitution.replace_applicator 0.19% : 0.000003s : 2: substitution.replace_old_param 0.38% : 0.000006s : 4: substitution.reset_defer_inline 0.30% : 0.000005s : 8: substitution.set_cell_output_no_recompute 0.16% : 0.000003s : 2: substitution.switch_call_monad_eliminater 0.81% : 0.000013s : 15: substitution.switch_defer_inline 1.30% : 0.000021s : 24: substitution.switch_simplify ------[renormalize.] 0.002170 2 75.31% : 0.001634s : 1: renormalize.infer 24.69% : 0.000536s : 1: renormalize.specialize ------[replace.] 0.000271 22 3.91% : 0.000011s : 2: replace.cast_eliminate 37.28% : 0.000101s : 6: replace.getattr_setattr_resolve 43.27% : 0.000117s : 13: replace.inline 15.53% : 0.000042s : 1: replace.switch_simplify ------[match.] 0.001327 22 1.27% : 0.000017s : 2: match.cast_eliminate 71.49% : 0.000949s : 6: match.getattr_setattr_resolve 27.07% : 0.000359s : 13: match.inline 0.17% : 0.000002s : 1: match.switch_simplify ------[func_graph_cloner_run.] 0.001246 24 33.20% : 0.000414s : 3: func_graph_cloner_run.FuncGraphClonerGraph 12.85% : 0.000160s : 2: func_graph_cloner_run.FuncGraphClonerNode 53.95% : 0.000672s : 19: func_graph_cloner_run.FuncGraphSpecializer ------[meta_graph.] 0.000000 0 ------[manager.] 0.000000 0 ------[pynative] 0.000000 0 ------[others.] 0.003178 105 14.03% : 0.000446s : 52: opt.transform.opt_a 4.76% : 0.000151s : 23: opt.transform.opt_b 35.84% : 0.001139s : 2: opt.transform.opt_resolve 2.19% : 0.000069s : 1: opt.transforms.meta_unpack_prepare 37.86% : 0.001203s : 20: opt.transforms.opt_a 1.09% : 0.000035s : 1: opt.transforms.opt_after_cconv 0.23% : 0.000007s : 1: opt.transforms.opt_b 3.01% : 0.000096s : 2: opt.transforms.opt_trans_graph 1.00% : 0.000032s : 3: opt.transforms.special_op_eliminate [TRACE] GE(61248,python3.7):2024-01-10-09:35:42.007.010 [status:INIT] [ge_api.cc:144]62979 GEInitializeImpl:GEInitialize start [INFO] GE(61248,python3.7):2024-01-10-09:35:42.007.305 [scalable_config.cc:55][EVENT]62979 ScalableConfig:device total max size: 34359738368, page_mem_size_total_thresold: 32641751449, uncacheable_size_threshold: 17179869184 [INFO] GE(61248,python3.7):2024-01-10-09:35:42.088.723 [graph_var_manager.cc:1424][EVENT]62979 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(61248,python3.7):2024-01-10-09:35:42.088.839 [graph_manager.cc:1248][EVENT]62979 PreRun:PreRun start: graph node size 4, session id 1, graph id 0, graph name online. [INFO] ATRACE(61248,python3.7):2024-01-10-09:35:42.089.765 [atrace_api.c:28](tid:62979) AtraceCreate start [INFO] ATRACE(61248,python3.7):2024-01-10-09:35:42.089.840 [trace_rb_log.c:84](tid:62979) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(61248,python3.7):2024-01-10-09:35:42.089.854 [atrace_api.c:32](tid:62979) AtraceCreate end [INFO] TDT(61248,python3.7):2024-01-10-09:35:42.089.874 [client_manager.cpp:157][SetProfilingCallback][tid:62979] [TsdClient] set profiling callback success [INFO] GE(61248,python3.7):2024-01-10-09:35:42.090.886 [parallel_partitioner.cc:165][EVENT]62979 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [28] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.090.929 [parallel_partitioner.cc:178][EVENT]62979 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.090.975 [graph_prepare.cc:1378][EVENT]62979 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.091.652 [graph_manager.cc:1050][EVENT]62979 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [693] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.091.679 [graph_manager.cc:1052][EVENT]62979 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.091.821 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.091.849 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [3] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.091.963 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [102] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.091.977 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.092.054 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [35] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.092.067 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [0] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.092.083 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [5] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.092.165 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.092.211 [graph_manager.cc:1054][EVENT]62979 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [518] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.098.650 [graph_manager.cc:1055][EVENT]62979 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [6425] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.100.014 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.100.043 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.100.055 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of MergePass is [27] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.100.064 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of InferShapePass is [342] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.100.073 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [47] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.100.081 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.100.090 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [115] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.100.098 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.100.106 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of InferValuePass is [10] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.101.848 [graph_manager.cc:1056][EVENT]62979 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3159] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.101.910 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.101.928 [graph_prepare.cc:1982][EVENT]62979 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [47] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.306 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.331 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.342 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.351 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of InferShapePass is [201] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.360 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.368 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.377 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.385 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.405 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.429 [graph_prepare.cc:1983][EVENT]62979 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [488] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.451 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.463 [graph_prepare.cc:1984][EVENT]62979 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [19] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.476 [graph_prepare.cc:1985][EVENT]62979 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [2] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.502 [graph_prepare.cc:1986][EVENT]62979 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [14] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.515 [graph_prepare.cc:1987][EVENT]62979 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.529 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.540 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.553 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [3] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.629 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.640 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.650 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.658 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [0] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.667 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of DropOutPass is [0] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.675 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.683 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [0] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.691 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.699 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.708 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.716 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.724 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.740 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.749 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [7] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.757 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.765 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.785 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [8] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.797 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.827 [graph_prepare.cc:1988][EVENT]62979 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [302] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.102.839 [graph_manager.cc:1065][EVENT]62979 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [957] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.115.830 [graph_manager.cc:1077][EVENT]62979 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12971] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.115.886 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.115.931 [graph_manager.cc:1080][EVENT]62979 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [63] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.066 [graph_manager.cc:1081][EVENT]62979 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [14119] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.121 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.138 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.150 [graph_manager.cc:1082][EVENT]62979 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [39] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.179 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.196 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.209 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [3] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.295 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [75] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.312 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [2] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.352 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [30] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.367 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [2] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.425 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [32] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.444 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.475 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [19] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.521 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [36] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.541 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [8] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.554 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.565 [graph_manager.cc:2700][EVENT]62979 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [391] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.699 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.713 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of AddNPass is [0] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.723 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.732 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.740 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.748 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of CastRemovePass is [8] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.757 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [1] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.765 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [5] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.773 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.781 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.789 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.797 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.805 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.813 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.821 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.840 [graph_manager.cc:2741][EVENT]62979 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [258] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.849 [graph_manager.cc:2752][EVENT]62979 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.868 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [2] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.879 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.899 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [9] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.912 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.923 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.935 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.957 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [12] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.971 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.985 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [2] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.130.996 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.021 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [15] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.034 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.050 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [7] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.063 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [2] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.073 [graph_manager.cc:2810][EVENT]62979 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [208] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.102 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.113 [graph_manager.cc:2821][EVENT]62979 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [32] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.135 [graph_manager.cc:1087][EVENT]62979 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [968] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.258 [graph_manager.cc:1088][EVENT]62979 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [110] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.294 [graph_manager.cc:1089][EVENT]62979 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [16] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.321 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.336 [graph_manager.cc:1097][EVENT]62979 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.356 [graph_manager.cc:3325][EVENT]62979 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.741 [engine_place.cc:144][EVENT]62979 Run:The time cost of AIcoreEngine::CheckSupported is [269] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.767 [engine_place.cc:144][EVENT]62979 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.776 [engine_place.cc:144][EVENT]62979 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.854 [graph_manager.cc:3351][EVENT]62979 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [486] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.874 [graph_manager.cc:3364][EVENT]62979 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.939 [engine_partitioner.cc:1139][EVENT]62979 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.131.954 [engine_partitioner.cc:1142][EVENT]62979 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.132.108 [engine_partitioner.cc:1148][EVENT]62979 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [145] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.132.148 [engine_partitioner.cc:1155][EVENT]62979 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [26] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.132.187 [engine_partitioner.cc:1164][EVENT]62979 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [28] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.132.220 [graph_manager.cc:3405][EVENT]62979 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [334] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.132.238 [graph_manager.cc:3412][EVENT]62979 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.298.958 [graph_manager.cc:3422][EVENT]62979 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [166707] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.005 [graph_manager.cc:3428][EVENT]62979 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.123 [graph_manager.cc:3467][EVENT]62979 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [98] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.141 [graph_manager.cc:3377][EVENT]62979 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [167256] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.156 [graph_manager.cc:1106][EVENT]62979 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [167805] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.169 [graph_manager.cc:1115][EVENT]62979 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.202 [graph_manager.cc:1130][EVENT]62979 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.233 [graph_manager.cc:1131][EVENT]62979 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [17] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.264 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [12] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.280 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [4] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.289 [graph_manager.cc:2837][EVENT]62979 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [40] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.376 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.389 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.398 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.407 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.415 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.424 [base_pass.cc:339][EVENT]62979 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [9] micro second, call num is [4] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.433 [graph_manager.cc:2864][EVENT]62979 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [129] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.444 [graph_manager.cc:2872][EVENT]62979 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [2] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.462 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.476 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.490 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [4] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.503 [compile_nodes_pass.cc:88][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [2] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.513 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.525 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [3] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.606 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [71] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.639 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [21] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.661 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.674 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.687 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.696 [graph_manager.cc:2927][EVENT]62979 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [236] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.715 [graph_manager.cc:2937][EVENT]62979 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [10] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.747 [graph_manager.cc:2943][EVENT]62979 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [24] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.299.761 [graph_manager.cc:2950][EVENT]62979 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [4] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.308.855 [graph_manager.cc:2958][EVENT]62979 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [45] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.308.906 [graph_manager.cc:1132][EVENT]62979 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [9658] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.309.036 [graph_manager.cc:1135][EVENT]62979 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [115] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.309.087 [graph_manager.cc:2975][EVENT]62979 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [30] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.309.249 [graph_manager.cc:2981][EVENT]62979 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [148] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.309.270 [pass_manager.cc:82][EVENT]62979 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.309.282 [graph_manager.cc:2986][EVENT]62979 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [15] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.309.291 [graph_manager.cc:1136][EVENT]62979 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [235] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.309.436 [graph_manager.cc:3555][EVENT]62979 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [117] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.309.510 [engine_partitioner.cc:1139][EVENT]62979 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.309.527 [engine_partitioner.cc:1142][EVENT]62979 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.309.666 [engine_partitioner.cc:1148][EVENT]62979 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [129] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.309.705 [engine_partitioner.cc:1155][EVENT]62979 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [24] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.309.765 [engine_partitioner.cc:1164][EVENT]62979 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.309.792 [graph_builder.cc:865][EVENT]62979 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [319] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.309.878 [graph_builder.cc:288][EVENT]62979 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [65] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.309.988 [graph_builder.cc:293][EVENT]62979 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [95] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.310.217 [model_builder.cc:1133][EVENT]62979 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [106] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.310.534 [block_mem_assigner.cc:4069][EVENT]65523 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.310.534 [block_mem_assigner.cc:4069][EVENT]65522 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.311.002 [graph_mem_assigner.cc:2166][EVENT]62979 SetInputOffset:[IMAS]AfterAssignMemory : online_0 memoffset[1536], memtype[2], theory_min[3072], zero_copy[1536], total_size[1536], no_reuse[1536], streams[1], topo_mode[DFS], mop[], io_reuse[0:0], alloc_mode[] [INFO] GE(61248,python3.7):2024-01-10-09:35:42.311.094 [model_builder.cc:1144][EVENT]62979 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [854] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.311.121 [model_builder.cc:1152][EVENT]62979 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [9] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.311.138 [model_builder.cc:1157][EVENT]62979 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [5] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.311.262 [model_builder.cc:1167][EVENT]62979 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [112] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.311.282 [model_builder.cc:1174][EVENT]62979 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.311.304 [model_builder.cc:1180][EVENT]62979 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [7] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.311.344 [model_builder.cc:1184][EVENT]62979 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [29] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.311.362 [graph_builder.cc:304][EVENT]62979 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1353] micro second. [INFO] RUNTIME(61248,python3.7):2024-01-10-09:35:42.311.597 [logger.cc:1071] 62979 ModelBindStream: model_id=576, stream_id=1857, flag=0. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.311.714 [task_generator.cc:804][EVENT]62979 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [4] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.311.785 [task_generator.cc:805][EVENT]62979 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [54] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.312.650 [task_generator.cc:814][EVENT]62979 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [850] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.312.665 [task_generator.cc:954][EVENT]62979 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [956] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.312.735 [task_generator.cc:967][EVENT]62979 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [26] micro second. [INFO] RUNTIME(61248,python3.7):2024-01-10-09:35:42.312.754 [logger.cc:1084] 62979 ModelUnbindStream: model_id=576, stream_id=1857, [INFO] GE(61248,python3.7):2024-01-10-09:35:42.312.797 [graph_builder.cc:310][EVENT]62979 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [1421] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.312.897 [graph_manager.cc:1152][EVENT]62979 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3584] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.312.914 [graph_manager.cc:1164][EVENT]62979 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.312.947 [graph_manager.cc:1271][EVENT]62979 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [222221] micro second. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.312.959 [graph_manager.cc:1272][EVENT]62979 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(61248,python3.7):2024-01-10-09:35:42.313.281 [atrace_api.c:93](tid:62979) AtraceDestroy start [INFO] ATRACE(61248,python3.7):2024-01-10-09:35:42.313.309 [atrace_api.c:95](tid:62979) AtraceDestroy end [INFO] GE(61248,python3.7):2024-01-10-09:35:42.313.898 [model_introduction.cc:236][EVENT]62979 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(61248,python3.7):2024-01-10-09:35:42.313.919 [model_introduction.cc:294][EVENT]62979 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(61248,python3.7):2024-01-10-09:35:42.313.931 [model_introduction.cc:366][EVENT]62979 Data:model io_info size:153 .. ============================== 2 passed in 22.81s ============================== [TRACE] GE(61248,python3.7):2024-01-10-09:35:44.166.499 [status:INIT] [ge_api.cc:463]61248 ~Session:Start to destruct session. [TRACE] GE(61248,python3.7):2024-01-10-09:35:44.166.556 [status:RUNNING] [ge_api.cc:475]61248 ~Session:Session id is 0 [TRACE] GE(61248,python3.7):2024-01-10-09:35:44.166.568 [status:RUNNING] [ge_api.cc:476]61248 ~Session:Destroying session [TRACE] GE(61248,python3.7):2024-01-10-09:35:44.167.619 [status:STOP] [ge_api.cc:491]61248 ~Session:Session Destructor finished [TRACE] GE(61248,python3.7):2024-01-10-09:35:44.167.652 [status:INIT] [ge_api.cc:301]61248 GEFinalize:GEFinalize start [INFO] GE(61248,python3.7):2024-01-10-09:35:44.167.702 [execution_runtime.cc:80][EVENT]61248 FinalizeExecutionRuntime:Execution runtime finalize begin. [INFO] GE(61248,python3.7):2024-01-10-09:35:44.167.719 [execution_runtime.cc:92][EVENT]61248 FinalizeExecutionRuntime:Execution runtime finalized. [TRACE] GE(61248,python3.7):2024-01-10-09:35:44.167.727 [status:RUNNING] [ge_api.cc:313]61248 GEFinalize:Finalizing environment [INFO] TUNE(61248,python3.7):2024-01-10-09:35:44.560.221 [cann_kb_pyfunc_mgr.cpp:127][CANNKB][Tid:61248]"CannKbPyfuncMgr: enter PyObjectDeinit function, reference_[1]" [INFO] TUNE(61248,python3.7):2024-01-10-09:35:44.560.280 [cann_kb_pyfunc_mgr.cpp:138][CANNKB][Tid:61248]"CannKbPyfuncMgr: PyObjectDeinit function end successfully!" [INFO] TDT(61248,python3.7):2024-01-10-09:35:44.563.173 [process_mode_manager.cpp:184][Close][tid:61248] [TsdClient] Close [deviceId=2][sessionId=1] hccp and computer enter [INFO] TDT(61248,python3.7):2024-01-10-09:35:44.563.211 [version_verify.cpp:112][SpecialFeatureCheck][tid:61248] VersionVerify: previous type[7], supported [INFO] TDT(61248,python3.7):2024-01-10-09:35:44.563.261 [process_mode_manager.cpp:192][Close][tid:61248] [TsdClient][deviceId=2] [sessionId=1] wait hccp and computer process close respond [INFO] TDT(61248,python3.7):2024-01-10-09:35:44.594.084 [process_mode_manager.cpp:197][Close][tid:61248] [TsdClient][logicDeviceId_=2]has recv close hccp and computer process respond [INFO] TDT(61248,python3.7):2024-01-10-09:35:44.594.105 [stub_process_mode_nowin.cpp:151][CloseInHost][tid:61248] enter into CloseInHost deviceid[2] [INFO] TDT(61248,python3.7):2024-01-10-09:35:44.594.143 [stub_process_mode_nowin.cpp:154][CloseInHost][tid:61248] host cpu not support [INFO] TDT(61248,python3.7):2024-01-10-09:35:44.594.190 [process_mode_manager.cpp:208][Close][tid:61248] [TsdClient][deviceId=2] [sessionId=1] close hccp and computer process success [INFO] ATRACE(61248,python3.7):2024-01-10-09:35:44.594.204 [atrace_api.c:93](tid:61248) AtraceDestroy start [INFO] ATRACE(61248,python3.7):2024-01-10-09:35:44.594.225 [atrace_api.c:95](tid:61248) AtraceDestroy end [INFO] PROFILING(61248,python3.7):2024-01-10-09:35:44.594.250 [msprofiler_impl.cpp:156] >>> (tid:61248) ProfNotifySetDevice called, is open: 0, devId: 2 [INFO] GE(61248,python3.7):2024-01-10-09:35:44.699.286 [gelib.cc:324][EVENT]61248 SystemFinalize:Online infer finalize GELib success. [TRACE] GE(61248,python3.7):2024-01-10-09:35:45.260.057 [status:STOP] [ge_api.cc:341]61248 GEFinalize:GEFinalize finished [INFO] RUNTIME(61248,python3.7):2024-01-10-09:35:47.602.188 [runtime.cc:1737] 61248 ~Runtime: deconstruct runtime.