============================= test session starts ============================== platform linux -- Python 3.7.5, pytest-5.4.3, py-1.11.0, pluggy-0.13.1 rootdir: /home/jenkins/mindspore/testcases/testcases/tests/st/heterogeneous, inifile: /home/jenkins/sault/virtual_test/virtualenv_005/sault/config/pytest.ini plugins: xdist-1.32.0, forked-1.4.0 [INFO] ATRACE(193333,python3.7):2024-01-10-16:29:17.277.674 [trace_attr.c:105](tid:193333) platform is 1. [INFO] ATRACE(193333,python3.7):2024-01-10-16:29:17.277.844 [trace_recorder.c:114](tid:193333) use root path: /home/jenkins/ascend/atrace [INFO] ATRACE(193333,python3.7):2024-01-10-16:29:17.277.868 [trace_signal.c:133](tid:193333) register signal handler for signo 2 succeed. [INFO] ATRACE(193333,python3.7):2024-01-10-16:29:17.277.879 [trace_signal.c:133](tid:193333) register signal handler for signo 15 succeed. [INFO] RUNTIME(193333,python3.7):2024-01-10-16:29:17.667.091 [runtime.cc:1159] 193333 GetAicoreNumByLevel: workingDev_=0 [INFO] RUNTIME(193333,python3.7):2024-01-10-16:29:17.667.161 [runtime.cc:4719] 193333 GetVisibleDevices: ASCEND_RT_VISIBLE_DEVICES param was not set collected 2 items test_heter_control.py [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.367.133 [process_mode_manager.cpp:109][OpenProcess][tid:193333] [ProcessModeManager] enter into open process deviceId[4] rankSize[0] [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.370.141 [process_mode_manager.cpp:379][InitTsdClient][tid:193333] [TsdClient] deviceId[4] begin to init hdc client [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.370.292 [version_verify.cpp:34][SetVersionInfo][tid:193333] VersionVerify: send client version to server [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.370.320 [version_verify.cpp:50][SetVersionInfo][tid:193333] send feature_info:{msg_type:35, features:{check before send aicpu package,}} [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.370.332 [version_verify.cpp:50][SetVersionInfo][tid:193333] send feature_info:{msg_type:37, features:{check before send open qs message,}} [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.370.547 [version_verify.cpp:66][PeerVersionCheck][tid:193333] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.370.561 [version_verify.cpp:87][ParseVersionInfo][tid:193333] VersionVerify: pass client version info success [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.370.570 [hdc_client.cpp:276][CheckHdcConnection][tid:193333] Service[2] create hdc success [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.370.582 [version_verify.cpp:120][SpecialFeatureCheck][tid:193333] VersionVerify: new type[35], supported [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.370.626 [process_mode_manager.cpp:748][GetDeviceCheckCode][tid:193333] [TsdClient][deviceId=4] [sessionId=1] wait package info respond [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.370.737 [process_mode_manager.cpp:379][InitTsdClient][tid:193333] [TsdClient] deviceId[4] begin to init hdc client [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.370.832 [version_verify.cpp:34][SetVersionInfo][tid:193333] VersionVerify: send client version to server [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.370.844 [version_verify.cpp:50][SetVersionInfo][tid:193333] send feature_info:{msg_type:35, features:{check before send aicpu package,}} [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.370.854 [version_verify.cpp:50][SetVersionInfo][tid:193333] send feature_info:{msg_type:37, features:{check before send open qs message,}} [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.370.976 [version_verify.cpp:66][PeerVersionCheck][tid:193333] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.370.988 [version_verify.cpp:87][ParseVersionInfo][tid:193333] VersionVerify: pass client version info success [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.370.996 [hdc_client.cpp:276][CheckHdcConnection][tid:193333] Service[2] create hdc success [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.371.006 [process_mode_manager.cpp:426][ConstructOpenMsg][tid:193333] [TsdClient] tsd get process sign successfully, procpid[193333] signSize[48] [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.371.038 [version_verify.cpp:112][SpecialFeatureCheck][tid:193333] VersionVerify: previous type[6], supported [INFO] TDT(193333,python3.7):2024-01-10-16:29:22.371.058 [process_mode_manager.cpp:126][OpenProcess][tid:193333] [ProcessModeManager] deviceId[4] sessionId[1] rankSize[0], wait sub process start respond [INFO] TDT(193333,python3.7):2024-01-10-16:29:23.039.809 [stub_process_mode_nowin.cpp:63][ProcessQueueForMdc][tid:193333] [TsdClient] it is unnecessary of current mode[0] chiptype[1] to grant queue auth to aicpusd [INFO] TDT(193333,python3.7):2024-01-10-16:29:23.039.861 [stub_process_mode_nowin.cpp:101][OpenInHost][tid:193333] enter into OpenInHost deviceid[4] [INFO] TDT(193333,python3.7):2024-01-10-16:29:23.039.872 [stub_process_mode_nowin.cpp:105][OpenInHost][tid:193333] host cpu not support [INFO] TDT(193333,python3.7):2024-01-10-16:29:23.039.880 [process_mode_manager.cpp:156][OpenProcess][tid:193333] [TsdClient][deviceId=4] [sessionId=1] start hccp and computer process success [INFO] RUNTIME(193333,python3.7):2024-01-10-16:29:23.042.241 [device.cc:340] 193333 Init: isDoubledie:0, topologytype:0 [INFO] RUNTIME(193333,python3.7):2024-01-10-16:29:23.059.398 [npu_driver.cc:5428] 194489 GetDeviceStatus: GetDeviceStatus status=1. [INFO] ATRACE(193333,python3.7):2024-01-10-16:29:23.059.473 [atrace_api.c:28](tid:193333) AtraceCreate start [INFO] ATRACE(193333,python3.7):2024-01-10-16:29:23.059.556 [trace_rb_log.c:84](tid:193333) [RUNTIME_ATRACE_DEV4_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(193333,python3.7):2024-01-10-16:29:23.059.570 [atrace_api.c:32](tid:193333) AtraceCreate end [INFO] TDT(193333,python3.7):2024-01-10-16:29:23.059.585 [client_manager.cpp:157][SetProfilingCallback][tid:193333] [TsdClient] set profiling callback success [TRACE] GE(193333,python3.7):2024-01-10-16:29:23.128.037 [status:INIT] [ge_api.cc:144]193333 GEInitializeImpl:GEInitialize start [INFO] PROFILING(193333,python3.7):2024-01-10-16:29:23.332.866 [msprofiler_impl.cpp:156] >>> (tid:193333) ProfNotifySetDevice called, is open: 1, devId: 4 [INFO] PROFILING(193333,python3.7):2024-01-10-16:29:23.332.981 [platform.cpp:38] >>> (tid:193333) Profiling platform version: 1.0. [INFO] PROFILING(193333,python3.7):2024-01-10-16:29:23.332.996 [ai_drv_dev_api.cpp:384] >>> (tid:193333) Succeeded to DrvGetApiVersion version: 0x72313 [TRACE] GE(193333,python3.7):2024-01-10-16:29:23.381.016 [status:RUNNING] [ge_api.cc:211]193333 GEInitializeImpl:Initializing environment [INFO] GE(193333,python3.7):2024-01-10-16:29:23.381.083 [gelib.cc:98][EVENT]193333 Initialize:[GEPERFTRACE] GE Init Start [INFO] GE(193333,python3.7):2024-01-10-16:29:23.381.347 [gelib.cc:307][EVENT]193333 SystemInitialize:Online infer init GELib success, device id :4 [INFO] DVPP(193333,python3.7):2024-01-10-16:29:23.715.273 [dvpp_engine.cc:41][ENGINE][Initialize:41][tid:193333]dvpp engine do not support [INFO] TUNE(193333,python3.7):2024-01-10-16:29:23.718.739 [cann_kb_pyfunc_mgr.cpp:72][CANNKB][Tid:193333]"CannKbPyfuncMgr: Enter PyObjectInit, reference_ is 0!" [INFO] TUNE(193333,python3.7):2024-01-10-16:29:23.718.780 [handle_manager.cpp:115][CANNKB][Tid:193333]"Start to run init functions to load dynamic python lib!" [INFO] TUNE(193333,python3.7):2024-01-10-16:29:23.718.833 [handle_manager.cpp:407][CANNKB][Tid:193333]"Init functions of loading dynamic python lib end!" [INFO] TUNE(193333,python3.7):2024-01-10-16:29:23.718.843 [cann_kb_pyfunc_mgr.cpp:24][CANNKB][Tid:193333]"CANN_KB_Py has already been initialized." [INFO] TUNE(193333,python3.7):2024-01-10-16:29:23.718.926 [cann_kb_pyfunc_mgr.cpp:117][CANNKB][Tid:193333]"CannKbPyfuncMgr: Run PyObjectInit successfully!" [INFO] HCCL(193333,python3.7):2024-01-10-16:29:36.210.201 [plugin_manager.cc:42][193333]hcom running normal mode. [INFO] DVPP(193333,python3.7):2024-01-10-16:29:36.210.869 [dvpp_engine.cc:92][ENGINE][GetOpsKernelInfoStores:92][tid:193333]dvpp ops kernel info store do not support [INFO] DVPP(193333,python3.7):2024-01-10-16:29:36.211.075 [dvpp_engine.cc:69][ENGINE][GetGraphOptimizerObjs:69][tid:193333]dvpp graph optimizer do not support [INFO] DVPP(193333,python3.7):2024-01-10-16:29:36.800.278 [dvpp_ops_kernel_builder.cc:48][ENGINE][Initialize:48][tid:193333]dvpp ops kernel builder do not support [INFO] GE(193333,python3.7):2024-01-10-16:29:36.809.351 [gelib.cc:169][EVENT]193333 Initialize:[GEPERFTRACE] The time cost of GELib::Initialize is [13428219] micro second. [TRACE] GE(193333,python3.7):2024-01-10-16:29:36.914.064 [status:STOP] [ge_api.cc:255]193333 GEInitializeImpl:GEInitialize finished [TRACE] GE(193333,python3.7):2024-01-10-16:29:36.914.231 [status:INIT] [ge_api.cc:398]193333 Session:Start to construct session. [TRACE] GE(193333,python3.7):2024-01-10-16:29:36.914.251 [status:RUNNING] [ge_api.cc:408]193333 Session:Creating session [INFO] GE(193333,python3.7):2024-01-10-16:29:36.914.682 [graph_var_manager.cc:1445][EVENT]193333 SetMemoryMallocSize:Total memory size is 34359738368 [INFO] GE(193333,python3.7):2024-01-10-16:29:36.914.699 [graph_var_manager.cc:1424][EVENT]193333 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] PROFILING(193333,python3.7):2024-01-10-16:29:36.915.039 [msprofiler_impl.cpp:156] >>> (tid:193333) ProfNotifySetDevice called, is open: 1, devId: 4 [TRACE] GE(193333,python3.7):2024-01-10-16:29:36.915.994 [status:RUNNING] [ge_api.cc:411]193333 Session:Session id is 0 [TRACE] GE(193333,python3.7):2024-01-10-16:29:36.916.016 [status:STOP] [ge_api.cc:420]193333 Session:Session Constructor finished [INFO] PROFILING(193333,python3.7):2024-01-10-16:29:36.926.055 [platform.cpp:38] >>> (tid:193333) Profiling platform version: 1.0. [INFO] PROFILING(193333,python3.7):2024-01-10-16:29:36.926.098 [ai_drv_dev_api.cpp:384] >>> (tid:193333) Succeeded to DrvGetApiVersion version: 0x72313 [TRACE] GE(193333,python3.7):2024-01-10-16:29:36.926.335 [status:INIT] [ge_api.cc:144]193333 GEInitializeImpl:GEInitialize start TotalTime = 14.6487, [20] [parse]: 0.01184 [symbol_resolve]: 0.0010153, [1] [Cycle 1]: 0.00096843, [1] [resolve]: 0.00095221 [combine_like_graphs]: 6.79982e-07 [graph_reusing]: 2.46998e-06 [meta_unpack_prepare]: 5.373e-05 [pre_cconv]: 3.03995e-06 [abstract_specialize]: 0.0317501 [pack_expand]: 1.604e-05 [auto_monad]: 0.00012604 [inline]: 1.64e-06 [pre_auto_parallel]: 1.069e-05 [pipeline_split]: 1.47999e-06 [optimize]: 0.0121103, [35] [py_interpret_to_execute]: 3.124e-05 [rewriter_before_opt_a]: 9.623e-05 [opt_a]: 0.0105321, [2] [Cycle 1]: 0.0032163, [30] [expand_dump_flag]: 4.25003e-06 [switch_simplify]: 7.1e-05 [a_1]: 0.00066881 [recompute_prepare]: 1.247e-05 [updatestate_depend_eliminate]: 1.539e-05 [updatestate_assign_eliminate]: 1.033e-05 [updatestate_loads_eliminate]: 9.49e-06 [parameter_eliminate]: 5.56e-06 [a_2]: 0.00013533 [accelerated_algorithm]: 1.174e-05 [pynative_shard]: 1.07003e-06 [auto_parallel]: 3.71999e-06 [parallel]: 1.23e-05 [merge_comm]: 8.2e-06 [allreduce_fusion]: 1.61998e-06 [virtual_dataset]: 7.52998e-06 [get_grad_eliminate_]: 5.76e-06 [virtual_output]: 5.72996e-06 [merge_forward]: 1.14e-05 [cell_reuse_recompute_pass]: 5.00004e-07 [cell_reuse_handle_not_recompute_node_pass]: 1.66e-05 [meta_fg_expand]: 7.62998e-06 [after_resolve]: 7.32002e-06 [a_after_grad]: 8.17003e-06 [renormalize]: 0.00185178 [real_op_eliminate]: 1.011e-05 [auto_monad_grad]: 6.33002e-06 [auto_monad_eliminator]: 2.362e-05 [cse]: 7.445e-05 [a_3]: 5.854e-05 [Cycle 2]: 0.00059688, [30] [expand_dump_flag]: 1.29e-06 [switch_simplify]: 1.044e-05 [a_1]: 0.00010235 [recompute_prepare]: 9.96998e-06 [updatestate_depend_eliminate]: 1.091e-05 [updatestate_assign_eliminate]: 9.07999e-06 [updatestate_loads_eliminate]: 8.90996e-06 [parameter_eliminate]: 2.60002e-06 [a_2]: 0.00012291 [accelerated_algorithm]: 9.15996e-06 [pynative_shard]: 1.02003e-06 [auto_parallel]: 3.04001e-06 [parallel]: 3.65998e-06 [merge_comm]: 1.89996e-06 [allreduce_fusion]: 1.40001e-06 [virtual_dataset]: 6.83998e-06 [get_grad_eliminate_]: 5.98002e-06 [virtual_output]: 6.02001e-06 [merge_forward]: 1.031e-05 [cell_reuse_recompute_pass]: 3.60014e-07 [cell_reuse_handle_not_recompute_node_pass]: 1.613e-05 [meta_fg_expand]: 6.39004e-06 [after_resolve]: 7e-06 [a_after_grad]: 7.71001e-06 [renormalize]: 7.99773e-08 [real_op_eliminate]: 5.94003e-06 [auto_monad_grad]: 2.01003e-06 [auto_monad_eliminator]: 1.895e-05 [cse]: 4.093e-05 [a_3]: 5.392e-05 [py_interpret_to_execute_after_opt_a]: 2.073e-05 [slice_cell_reuse_recomputed_activation]: 1.41998e-06 [rewriter_after_opt_a]: 0.00030348 [convert_after_rewriter]: 1.686e-05 [order_py_execute_after_rewriter]: 1.159e-05 [opt_b]: 0.00026663, [1] [Cycle 1]: 0.00026172, [7] [b_1]: 0.00015335 [b_2]: 7.93004e-06 [updatestate_depend_eliminate]: 1.062e-05 [updatestate_assign_eliminate]: 9.02999e-06 [updatestate_loads_eliminate]: 8.75996e-06 [renormalize]: 2.70025e-07 [cse]: 4.305e-05 [cconv]: 0.00018532 [opt_after_cconv]: 0.00018309, [1] [Cycle 1]: 0.0001784, [7] [c_1]: 3.034e-05 [parameter_eliminate]: 5.51e-06 [updatestate_depend_eliminate]: 1.432e-05 [updatestate_assign_eliminate]: 1.168e-05 [updatestate_loads_eliminate]: 1.229e-05 [cse]: 7.404e-05 [renormalize]: 3.70026e-07 [remove_dup_value]: 5.356e-05 [tuple_transform]: 0.00010467, [1] [Cycle 1]: 0.00010074, [3] [d_1]: 6.016e-05 [d_2]: 2.809e-05 [renormalize]: 2.19967e-07 [add_cache_embedding]: 2.369e-05 [add_recomputation]: 6.828e-05 [cse_after_recomputation]: 5.74e-05, [1] [Cycle 1]: 5.313e-05, [1] [cse]: 4.803e-05 [environ_conv]: 1.645e-05 [label_micro_interleaved_index]: 1.75001e-06 [label_fine_grained_interleaved_index]: 1.49e-06 [assign_add_opt]: 1.49e-06 [slice_recompute_activation]: 1.39e-06 [micro_interleaved_order_control]: 1.10001e-06 [full_micro_interleaved_order_control]: 1.15001e-06 [comp_comm_scheduling]: 1.19995e-06 [reorder_send_recv_between_fp_bp]: 1.45996e-06 [comm_op_add_attrs]: 5.69969e-07 [add_comm_op_reuse_tag]: 5.19969e-07 [overlap_opt_shard_in_pipeline]: 5.39992e-07 [grouped_pairwise_exchange_alltoall]: 6.60017e-07 [overlap_recompute_and_grad_model_parallel]: 1.31002e-06 [overlap_grad_matmul_and_grad_allreduce]: 4.60015e-07 [split_matmul_comm_elemetwise]: 1.80997e-06 [split_layernorm_comm]: 1.49e-06 [process_send_recv_for_ge]: 1.90001e-06 [handle_group_info]: 8.89995e-07 [auto_monad_reorder]: 4.888e-05 [get_jit_bprop_graph]: 5.90051e-07 [eliminate_special_op_node]: 0.00047024 [validate]: 5.663e-05 [distribtued_split]: 8.89995e-07 [task_emit]: 14.5909 [execute]: 1.04e-05 Sums parse : 0.011840s : 0.08% symbol_resolve.resolve : 0.000952s : 0.01% combine_like_graphs : 0.000001s : 0.00% graph_reusing : 0.000002s : 0.00% meta_unpack_prepare : 0.000054s : 0.00% pre_cconv : 0.000003s : 0.00% abstract_specialize : 0.031750s : 0.22% pack_expand : 0.000016s : 0.00% auto_monad : 0.000126s : 0.00% inline : 0.000002s : 0.00% pre_auto_parallel : 0.000011s : 0.00% pipeline_split : 0.000001s : 0.00% optimize.py_interpret_to_execute : 0.000031s : 0.00% optimize.rewriter_before_opt_a : 0.000096s : 0.00% optimize.opt_a.expand_dump_flag : 0.000006s : 0.00% optimize.opt_a.switch_simplify : 0.000081s : 0.00% optimize.opt_a.a_1 : 0.000771s : 0.01% optimize.opt_a.recompute_prepare : 0.000022s : 0.00% optimize.opt_a.updatestate_depend_eliminate : 0.000026s : 0.00% optimize.opt_a.updatestate_assign_eliminate : 0.000019s : 0.00% optimize.opt_a.updatestate_loads_eliminate : 0.000018s : 0.00% optimize.opt_a.parameter_eliminate : 0.000008s : 0.00% optimize.opt_a.a_2 : 0.000258s : 0.00% optimize.opt_a.accelerated_algorithm : 0.000021s : 0.00% optimize.opt_a.pynative_shard : 0.000002s : 0.00% optimize.opt_a.auto_parallel : 0.000007s : 0.00% optimize.opt_a.parallel : 0.000016s : 0.00% optimize.opt_a.merge_comm : 0.000010s : 0.00% optimize.opt_a.allreduce_fusion : 0.000003s : 0.00% optimize.opt_a.virtual_dataset : 0.000014s : 0.00% optimize.opt_a.get_grad_eliminate_ : 0.000012s : 0.00% optimize.opt_a.virtual_output : 0.000012s : 0.00% optimize.opt_a.merge_forward : 0.000022s : 0.00% optimize.opt_a.cell_reuse_recompute_pass : 0.000001s : 0.00% optimize.opt_a.cell_reuse_handle_not_recompute_node_pass : 0.000033s : 0.00% optimize.opt_a.meta_fg_expand : 0.000014s : 0.00% optimize.opt_a.after_resolve : 0.000014s : 0.00% optimize.opt_a.a_after_grad : 0.000016s : 0.00% optimize.opt_a.renormalize : 0.001852s : 0.01% optimize.opt_a.real_op_eliminate : 0.000016s : 0.00% optimize.opt_a.auto_monad_grad : 0.000008s : 0.00% optimize.opt_a.auto_monad_eliminator : 0.000043s : 0.00% optimize.opt_a.cse : 0.000115s : 0.00% optimize.opt_a.a_3 : 0.000112s : 0.00% optimize.py_interpret_to_execute_after_opt_a : 0.000021s : 0.00% optimize.slice_cell_reuse_recomputed_activation : 0.000001s : 0.00% optimize.rewriter_after_opt_a : 0.000303s : 0.00% optimize.convert_after_rewriter : 0.000017s : 0.00% optimize.order_py_execute_after_rewriter : 0.000012s : 0.00% optimize.opt_b.b_1 : 0.000153s : 0.00% optimize.opt_b.b_2 : 0.000008s : 0.00% optimize.opt_b.updatestate_depend_eliminate : 0.000011s : 0.00% optimize.opt_b.updatestate_assign_eliminate : 0.000009s : 0.00% optimize.opt_b.updatestate_loads_eliminate : 0.000009s : 0.00% optimize.opt_b.renormalize : 0.000000s : 0.00% optimize.opt_b.cse : 0.000043s : 0.00% optimize.cconv : 0.000185s : 0.00% optimize.opt_after_cconv.c_1 : 0.000030s : 0.00% optimize.opt_after_cconv.parameter_eliminate : 0.000006s : 0.00% optimize.opt_after_cconv.updatestate_depend_eliminate : 0.000014s : 0.00% optimize.opt_after_cconv.updatestate_assign_eliminate : 0.000012s : 0.00% optimize.opt_after_cconv.updatestate_loads_eliminate : 0.000012s : 0.00% optimize.opt_after_cconv.cse : 0.000074s : 0.00% optimize.opt_after_cconv.renormalize : 0.000000s : 0.00% optimize.remove_dup_value : 0.000054s : 0.00% optimize.tuple_transform.d_1 : 0.000060s : 0.00% optimize.tuple_transform.d_2 : 0.000028s : 0.00% optimize.tuple_transform.renormalize : 0.000000s : 0.00% optimize.add_cache_embedding : 0.000024s : 0.00% optimize.add_recomputation : 0.000068s : 0.00% optimize.cse_after_recomputation.cse : 0.000048s : 0.00% optimize.environ_conv : 0.000016s : 0.00% optimize.label_micro_interleaved_index : 0.000002s : 0.00% optimize.label_fine_grained_interleaved_index : 0.000001s : 0.00% optimize.assign_add_opt : 0.000001s : 0.00% optimize.slice_recompute_activation : 0.000001s : 0.00% optimize.micro_interleaved_order_control : 0.000001s : 0.00% optimize.full_micro_interleaved_order_control : 0.000001s : 0.00% optimize.comp_comm_scheduling : 0.000001s : 0.00% optimize.reorder_send_recv_between_fp_bp : 0.000001s : 0.00% optimize.comm_op_add_attrs : 0.000001s : 0.00% optimize.add_comm_op_reuse_tag : 0.000001s : 0.00% optimize.overlap_opt_shard_in_pipeline : 0.000001s : 0.00% optimize.grouped_pairwise_exchange_alltoall : 0.000001s : 0.00% optimize.overlap_recompute_and_grad_model_parallel : 0.000001s : 0.00% optimize.overlap_grad_matmul_and_grad_allreduce : 0.000000s : 0.00% optimize.split_matmul_comm_elemetwise : 0.000002s : 0.00% optimize.split_layernorm_comm : 0.000001s : 0.00% optimize.process_send_recv_for_ge : 0.000002s : 0.00% optimize.handle_group_info : 0.000001s : 0.00% auto_monad_reorder : 0.000049s : 0.00% get_jit_bprop_graph : 0.000001s : 0.00% eliminate_special_op_node : 0.000470s : 0.00% validate : 0.000057s : 0.00% distribtued_split : 0.000001s : 0.00% task_emit : 14.590934s : 99.66% execute : 0.000010s : 0.00% Time group info: ------[substitution.] 0.001224 245 3.51% : 0.000043s : 12: substitution.arithmetic_simplify 0.76% : 0.000009s : 2: substitution.cast_eliminate 0.74% : 0.000009s : 4: substitution.compare_switch_simplify 0.87% : 0.000011s : 15: substitution.exchange_switch_depend_value 0.16% : 0.000002s : 4: substitution.float_depend_g_call 64.23% : 0.000786s : 6: substitution.getattr_setattr_resolve 1.55% : 0.000019s : 27: substitution.graph_param_transform 0.23% : 0.000003s : 4: substitution.incorporate_call 0.11% : 0.000001s : 4: substitution.incorporate_call_switch 21.33% : 0.000261s : 13: substitution.inline 0.17% : 0.000002s : 2: substitution.less_batch_normalization 1.25% : 0.000015s : 20: substitution.meta_unpack_prepare 0.77% : 0.000009s : 6: substitution.partial_eliminate 0.42% : 0.000005s : 27: substitution.partial_unused_args_eliminate 0.25% : 0.000003s : 28: substitution.remove_not_recompute_node 0.78% : 0.000010s : 16: substitution.replace_applicator 0.12% : 0.000001s : 2: substitution.replace_old_param 0.29% : 0.000004s : 4: substitution.reset_defer_inline 0.31% : 0.000004s : 8: substitution.set_cell_output_no_recompute 0.08% : 0.000001s : 2: substitution.switch_call_monad_eliminater 0.80% : 0.000010s : 15: substitution.switch_defer_inline 1.27% : 0.000016s : 24: substitution.switch_simplify ------[renormalize.] 0.001848 2 75.97% : 0.001404s : 1: renormalize.infer 24.03% : 0.000444s : 1: renormalize.specialize ------[replace.] 0.000225 22 3.92% : 0.000009s : 2: replace.cast_eliminate 38.56% : 0.000087s : 6: replace.getattr_setattr_resolve 45.43% : 0.000102s : 13: replace.inline 12.09% : 0.000027s : 1: replace.switch_simplify ------[match.] 0.001057 22 0.88% : 0.000009s : 2: match.cast_eliminate 74.32% : 0.000786s : 6: match.getattr_setattr_resolve 24.68% : 0.000261s : 13: match.inline 0.13% : 0.000001s : 1: match.switch_simplify ------[func_graph_cloner_run.] 0.000902 24 36.37% : 0.000328s : 3: func_graph_cloner_run.FuncGraphClonerGraph 11.71% : 0.000106s : 2: func_graph_cloner_run.FuncGraphClonerNode 51.92% : 0.000468s : 19: func_graph_cloner_run.FuncGraphSpecializer ------[meta_graph.] 0.000000 0 ------[manager.] 0.000000 0 ------[pynative] 0.000000 0 ------[others.] 0.002624 105 14.58% : 0.000383s : 52: opt.transform.opt_a 5.49% : 0.000144s : 23: opt.transform.opt_b 35.83% : 0.000940s : 2: opt.transform.opt_resolve 1.49% : 0.000039s : 1: opt.transforms.meta_unpack_prepare 36.82% : 0.000966s : 20: opt.transforms.opt_a 1.11% : 0.000029s : 1: opt.transforms.opt_after_cconv 0.26% : 0.000007s : 1: opt.transforms.opt_b 3.28% : 0.000086s : 2: opt.transforms.opt_trans_graph 1.13% : 0.000030s : 3: opt.transforms.special_op_eliminate [INFO] GE(193333,python3.7):2024-01-10-16:29:36.948.343 [scalable_config.cc:55][EVENT]194438 ScalableConfig:device total max size: 34359738368, page_mem_size_total_thresold: 32641751449, uncacheable_size_threshold: 17179869184 [INFO] GE(193333,python3.7):2024-01-10-16:29:37.027.209 [graph_var_manager.cc:1424][EVENT]194438 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(193333,python3.7):2024-01-10-16:29:37.027.299 [graph_manager.cc:1248][EVENT]194438 PreRun:PreRun start: graph node size 4, session id 1, graph id 0, graph name online. [INFO] ATRACE(193333,python3.7):2024-01-10-16:29:37.028.218 [atrace_api.c:28](tid:194438) AtraceCreate start [INFO] ATRACE(193333,python3.7):2024-01-10-16:29:37.028.293 [trace_rb_log.c:84](tid:194438) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(193333,python3.7):2024-01-10-16:29:37.028.306 [atrace_api.c:32](tid:194438) AtraceCreate end [INFO] TDT(193333,python3.7):2024-01-10-16:29:37.028.332 [client_manager.cpp:157][SetProfilingCallback][tid:194438] [TsdClient] set profiling callback success [INFO] GE(193333,python3.7):2024-01-10-16:29:37.029.281 [parallel_partitioner.cc:165][EVENT]194438 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [24] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.029.324 [parallel_partitioner.cc:178][EVENT]194438 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.029.370 [graph_prepare.cc:1378][EVENT]194438 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [2] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.030.066 [graph_manager.cc:1050][EVENT]194438 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [712] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.030.094 [graph_manager.cc:1052][EVENT]194438 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.030.222 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.030.248 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [3] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.030.309 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [49] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.030.322 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.030.368 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [13] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.030.398 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.030.417 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.030.505 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.030.525 [graph_manager.cc:1054][EVENT]194438 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [418] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.036.914 [graph_manager.cc:1055][EVENT]194438 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [6375] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.037.920 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.037.948 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.037.959 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.037.969 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of InferShapePass is [317] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.037.978 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.037.987 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.037.995 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [18] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.038.003 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [21] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.038.011 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of InferValuePass is [15] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.039.582 [graph_manager.cc:1056][EVENT]194438 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2633] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.039.641 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.039.659 [graph_prepare.cc:1982][EVENT]194438 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [46] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.039 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [8] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.063 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.074 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.083 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of InferShapePass is [203] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.092 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.113 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [8] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.122 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.131 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.139 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.162 [graph_prepare.cc:1983][EVENT]194438 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [489] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.184 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [3] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.196 [graph_prepare.cc:1984][EVENT]194438 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [19] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.210 [graph_prepare.cc:1985][EVENT]194438 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [2] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.240 [graph_prepare.cc:1986][EVENT]194438 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [16] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.253 [graph_prepare.cc:1987][EVENT]194438 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.268 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.281 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.295 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.372 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.384 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.393 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.402 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.411 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.419 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.427 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.435 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [0] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.443 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.463 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [0] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.472 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.480 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.488 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.496 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [2] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.504 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.512 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.532 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [8] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.545 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.577 [graph_prepare.cc:1988][EVENT]194438 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [313] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.040.589 [graph_manager.cc:1065][EVENT]194438 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [975] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.053.121 [graph_manager.cc:1077][EVENT]194438 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12509] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.053.195 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.053.241 [graph_manager.cc:1080][EVENT]194438 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [83] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.067.989 [graph_manager.cc:1081][EVENT]194438 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [14731] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.043 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.059 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [1] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.072 [graph_manager.cc:1082][EVENT]194438 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [38] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.099 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.114 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [2] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.128 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [2] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.208 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [70] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.239 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [2] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.283 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [32] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.299 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [2] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.344 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [34] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.365 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.393 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [18] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.445 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [40] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.464 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.476 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [1] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.487 [graph_manager.cc:2700][EVENT]194438 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [393] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.615 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.630 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.640 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.649 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.657 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.666 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of CastRemovePass is [8] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.674 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.682 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.690 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.698 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.706 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.714 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.731 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.740 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.748 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [4] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.759 [graph_manager.cc:2741][EVENT]194438 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [255] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.770 [graph_manager.cc:2752][EVENT]194438 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.791 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.804 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.822 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.835 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [3] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.847 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [1] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.859 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.875 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [6] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.888 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [3] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.900 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [2] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.910 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.938 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [18] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.950 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.966 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [6] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.978 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [2] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.068.988 [graph_manager.cc:2810][EVENT]194438 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [202] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.069.017 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.069.030 [graph_manager.cc:2821][EVENT]194438 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [33] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.069.057 [graph_manager.cc:1087][EVENT]194438 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [968] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.069.201 [graph_manager.cc:1088][EVENT]194438 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [130] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.069.246 [graph_manager.cc:1089][EVENT]194438 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [21] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.069.265 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.069.280 [graph_manager.cc:1097][EVENT]194438 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.069.299 [graph_manager.cc:3325][EVENT]194438 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.069.690 [engine_place.cc:144][EVENT]194438 Run:The time cost of AIcoreEngine::CheckSupported is [272] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.069.717 [engine_place.cc:144][EVENT]194438 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.069.726 [engine_place.cc:144][EVENT]194438 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.069.817 [graph_manager.cc:3351][EVENT]194438 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [505] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.069.839 [graph_manager.cc:3364][EVENT]194438 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [1] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.069.905 [engine_partitioner.cc:1139][EVENT]194438 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.069.923 [engine_partitioner.cc:1142][EVENT]194438 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.070.064 [engine_partitioner.cc:1148][EVENT]194438 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [132] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.070.101 [engine_partitioner.cc:1155][EVENT]194438 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [22] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.070.143 [engine_partitioner.cc:1164][EVENT]194438 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [30] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.070.169 [graph_manager.cc:3405][EVENT]194438 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [318] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.070.188 [graph_manager.cc:3412][EVENT]194438 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.241.478 [graph_manager.cc:3422][EVENT]194438 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [171274] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.241.553 [graph_manager.cc:3428][EVENT]194438 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [14] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.241.718 [graph_manager.cc:3467][EVENT]194438 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [122] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.241.737 [graph_manager.cc:3377][EVENT]194438 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [171885] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.241.751 [graph_manager.cc:1106][EVENT]194438 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [172458] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.241.763 [graph_manager.cc:1115][EVENT]194438 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.241.786 [graph_manager.cc:1130][EVENT]194438 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.241.820 [graph_manager.cc:1131][EVENT]194438 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [21] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.241.849 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [10] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.241.865 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.241.876 [graph_manager.cc:2837][EVENT]194438 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [40] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.241.973 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.241.987 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.241.996 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.005 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.013 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.022 [base_pass.cc:339][EVENT]194438 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [12] micro second, call num is [4] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.031 [graph_manager.cc:2864][EVENT]194438 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [140] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.041 [graph_manager.cc:2872][EVENT]194438 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [2] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.060 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.073 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.089 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [4] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.102 [compile_nodes_pass.cc:88][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [2] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.121 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [22] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.135 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [4] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.223 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [77] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.265 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [28] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.279 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [1] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.294 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.307 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.317 [graph_manager.cc:2927][EVENT]194438 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [260] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.335 [graph_manager.cc:2937][EVENT]194438 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [9] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.372 [graph_manager.cc:2943][EVENT]194438 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [27] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.242.387 [graph_manager.cc:2950][EVENT]194438 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [4] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.251.446 [graph_manager.cc:2958][EVENT]194438 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [45] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.251.497 [graph_manager.cc:1132][EVENT]194438 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [9663] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.251.635 [graph_manager.cc:1135][EVENT]194438 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [122] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.251.677 [graph_manager.cc:2975][EVENT]194438 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [22] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.251.809 [graph_manager.cc:2981][EVENT]194438 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [117] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.251.828 [pass_manager.cc:82][EVENT]194438 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.251.839 [graph_manager.cc:2986][EVENT]194438 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [15] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.251.848 [graph_manager.cc:1136][EVENT]194438 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [194] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.251.991 [graph_manager.cc:3555][EVENT]194438 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [110] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.252.065 [engine_partitioner.cc:1139][EVENT]194438 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.252.094 [engine_partitioner.cc:1142][EVENT]194438 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.252.236 [engine_partitioner.cc:1148][EVENT]194438 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [131] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.252.271 [engine_partitioner.cc:1155][EVENT]194438 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [21] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.252.315 [engine_partitioner.cc:1164][EVENT]194438 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [32] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.252.340 [graph_builder.cc:865][EVENT]194438 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [311] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.252.431 [graph_builder.cc:288][EVENT]194438 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [67] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.252.543 [graph_builder.cc:293][EVENT]194438 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [98] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.252.782 [model_builder.cc:1133][EVENT]194438 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [110] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.253.147 [block_mem_assigner.cc:4069][EVENT]1922 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.253.150 [block_mem_assigner.cc:4069][EVENT]1923 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.253.618 [graph_mem_assigner.cc:2166][EVENT]194438 SetInputOffset:[IMAS]AfterAssignMemory : online_0 memoffset[1536], memtype[2], theory_min[3072], zero_copy[1536], total_size[1536], no_reuse[1536], streams[1], topo_mode[DFS], mop[], io_reuse[0:0], alloc_mode[] [INFO] GE(193333,python3.7):2024-01-10-16:29:37.253.707 [model_builder.cc:1144][EVENT]194438 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [901] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.253.734 [model_builder.cc:1152][EVENT]194438 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [8] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.253.751 [model_builder.cc:1157][EVENT]194438 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [4] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.253.871 [model_builder.cc:1167][EVENT]194438 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [107] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.253.891 [model_builder.cc:1174][EVENT]194438 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.253.913 [model_builder.cc:1180][EVENT]194438 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [7] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.253.946 [model_builder.cc:1184][EVENT]194438 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [21] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.253.963 [graph_builder.cc:304][EVENT]194438 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1399] micro second. [INFO] RUNTIME(193333,python3.7):2024-01-10-16:29:37.254.203 [logger.cc:1071] 194438 ModelBindStream: model_id=576, stream_id=1857, flag=0. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.254.337 [task_generator.cc:804][EVENT]194438 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [4] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.254.414 [task_generator.cc:805][EVENT]194438 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [61] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.255.232 [task_generator.cc:814][EVENT]194438 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [802] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.255.247 [task_generator.cc:954][EVENT]194438 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [914] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.255.291 [task_generator.cc:967][EVENT]194438 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [22] micro second. [INFO] RUNTIME(193333,python3.7):2024-01-10-16:29:37.255.309 [logger.cc:1084] 194438 ModelUnbindStream: model_id=576, stream_id=1857, [INFO] GE(193333,python3.7):2024-01-10-16:29:37.255.347 [graph_builder.cc:310][EVENT]194438 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [1371] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.255.446 [graph_manager.cc:1152][EVENT]194438 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3574] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.255.464 [graph_manager.cc:1164][EVENT]194438 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.255.494 [graph_manager.cc:1271][EVENT]194438 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [226337] micro second. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.255.506 [graph_manager.cc:1272][EVENT]194438 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(193333,python3.7):2024-01-10-16:29:37.255.817 [atrace_api.c:93](tid:194438) AtraceDestroy start [INFO] ATRACE(193333,python3.7):2024-01-10-16:29:37.255.842 [atrace_api.c:95](tid:194438) AtraceDestroy end [INFO] GE(193333,python3.7):2024-01-10-16:29:37.256.432 [model_introduction.cc:236][EVENT]194438 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(193333,python3.7):2024-01-10-16:29:37.256.453 [model_introduction.cc:294][EVENT]194438 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(193333,python3.7):2024-01-10-16:29:37.256.465 [model_introduction.cc:366][EVENT]194438 Data:model io_info size:153 .. ============================== 2 passed in 21.99s ============================== [TRACE] GE(193333,python3.7):2024-01-10-16:29:39.078.570 [status:INIT] [ge_api.cc:463]193333 ~Session:Start to destruct session. [TRACE] GE(193333,python3.7):2024-01-10-16:29:39.078.645 [status:RUNNING] [ge_api.cc:475]193333 ~Session:Session id is 0 [TRACE] GE(193333,python3.7):2024-01-10-16:29:39.078.656 [status:RUNNING] [ge_api.cc:476]193333 ~Session:Destroying session [TRACE] GE(193333,python3.7):2024-01-10-16:29:39.079.744 [status:STOP] [ge_api.cc:491]193333 ~Session:Session Destructor finished [TRACE] GE(193333,python3.7):2024-01-10-16:29:39.079.782 [status:INIT] [ge_api.cc:301]193333 GEFinalize:GEFinalize start [INFO] GE(193333,python3.7):2024-01-10-16:29:39.079.836 [execution_runtime.cc:80][EVENT]193333 FinalizeExecutionRuntime:Execution runtime finalize begin. [INFO] GE(193333,python3.7):2024-01-10-16:29:39.079.855 [execution_runtime.cc:92][EVENT]193333 FinalizeExecutionRuntime:Execution runtime finalized. [TRACE] GE(193333,python3.7):2024-01-10-16:29:39.079.864 [status:RUNNING] [ge_api.cc:313]193333 GEFinalize:Finalizing environment [INFO] TUNE(193333,python3.7):2024-01-10-16:29:39.426.342 [cann_kb_pyfunc_mgr.cpp:127][CANNKB][Tid:193333]"CannKbPyfuncMgr: enter PyObjectDeinit function, reference_[1]" [INFO] TUNE(193333,python3.7):2024-01-10-16:29:39.426.417 [cann_kb_pyfunc_mgr.cpp:138][CANNKB][Tid:193333]"CannKbPyfuncMgr: PyObjectDeinit function end successfully!" [INFO] TDT(193333,python3.7):2024-01-10-16:29:39.430.035 [process_mode_manager.cpp:184][Close][tid:193333] [TsdClient] Close [deviceId=4][sessionId=1] hccp and computer enter [INFO] TDT(193333,python3.7):2024-01-10-16:29:39.430.083 [version_verify.cpp:112][SpecialFeatureCheck][tid:193333] VersionVerify: previous type[7], supported [INFO] TDT(193333,python3.7):2024-01-10-16:29:39.430.147 [process_mode_manager.cpp:192][Close][tid:193333] [TsdClient][deviceId=4] [sessionId=1] wait hccp and computer process close respond [INFO] TDT(193333,python3.7):2024-01-10-16:29:39.450.972 [process_mode_manager.cpp:197][Close][tid:193333] [TsdClient][logicDeviceId_=4]has recv close hccp and computer process respond [INFO] TDT(193333,python3.7):2024-01-10-16:29:39.451.010 [stub_process_mode_nowin.cpp:151][CloseInHost][tid:193333] enter into CloseInHost deviceid[4] [INFO] TDT(193333,python3.7):2024-01-10-16:29:39.451.021 [stub_process_mode_nowin.cpp:154][CloseInHost][tid:193333] host cpu not support [INFO] TDT(193333,python3.7):2024-01-10-16:29:39.451.078 [process_mode_manager.cpp:208][Close][tid:193333] [TsdClient][deviceId=4] [sessionId=1] close hccp and computer process success [INFO] ATRACE(193333,python3.7):2024-01-10-16:29:39.451.094 [atrace_api.c:93](tid:193333) AtraceDestroy start [INFO] ATRACE(193333,python3.7):2024-01-10-16:29:39.451.783 [atrace_api.c:95](tid:193333) AtraceDestroy end [INFO] PROFILING(193333,python3.7):2024-01-10-16:29:39.451.825 [msprofiler_impl.cpp:156] >>> (tid:193333) ProfNotifySetDevice called, is open: 0, devId: 4 [INFO] GE(193333,python3.7):2024-01-10-16:29:39.563.628 [gelib.cc:324][EVENT]193333 SystemFinalize:Online infer finalize GELib success. [TRACE] GE(193333,python3.7):2024-01-10-16:29:39.937.581 [status:STOP] [ge_api.cc:341]193333 GEFinalize:GEFinalize finished [INFO] RUNTIME(193333,python3.7):2024-01-10-16:29:42.885.406 [runtime.cc:1737] 193333 ~Runtime: deconstruct runtime.